Vector Quantizer Design Using Genetic Algorithms

The design of vector quantizers (VQs) that yield minimal distortion is one of the most challenging problems in source coding. The problem of VQ design is to find a codebook that gives the least overall distortion (or equivalently, the largest signal-to-noise ratio (SNR)) for a given set of input vectors. This problem is known to be difficult as there are no known closed-form solutions. The Generalized Lloyd Algorithm (GLA) [I] uses a finite set of training sequences as the data source and employs an iterative refinement. Given an initial codebook, the algorithm computes the nearest locally optimum codebook only. Genetic algorithms (GAS) [2] are emerging as widely accepted optimization and search methods. These search methods are rooted in the mechanisms of evolution and natural genetics. They have a high probability of locating the globally optimal solution in a multimodal search space. A Genetic Algorithmic (GA) app roach to vector quantizer design that combines the GLA is presented. We refer to this hybrid as the Genetic Generalized Lloyd Algorithm (GGLA). It works briefly as follows: Initially, a finite number of codebooks, called chromosomes, are selected. In contrast to the GLA which refines only one codebook at a time, those codebooks undergo iterative cycles of reproduction in parallel. During an iteration, each codebook is updated by GLA or GA operations (i.e., mutation, CTOSSO~~, and chromosome replacement). Three versions of the GGLAs are investigated depending on how GLA or GA is selected. We make a fair comparison of the GGLAs with the GLA under the same design conditions using Gaussian-Markov sources, speech, and image as source data and signal-to-noise ratio (SNR) (and peak SNR for images) as the performance measure. For Gaussian-Markov sources, up to 0.6 dB improvement is observed using the GGLAs while for the image source, there is an improvement of no more than 0.2 dB. In general, larger improvement is observed for higher code rate. It is observed that the GGLAs do not perform well for the speech source. We conclude that in most cases, the GGLAs show performance improvements with respect to the GLA at the cost of more memory size needed for parallel operations. We finally compare our results with the Zador-Gersho formula, which is a we&known benchmark for VQ performance. A full exposition of this paper is available in postscript by sending an email to the first author. Parts of this work also appeared in [3]. [l] Y. LINDE, A. Buzo, R. M. GRAY. An Algorithm for Vector Quantizer Design. IEEE Trans. on Communicalions, Vol. 28, No. 1, pp. 84-95, January 1980. [2] D. E. GOLDBERG. Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley, Reading, Massachusetts, 1989. [3] WEE-KEONG NG, SUNGHYUN CHOI, CHINYA RAVISHANKAR. An Evolutionary Approach to Vector Quantizer Design. Proceedings of the 2nd IEEE International Conference on Euohticmary Computing [ICEC’95), Perth, Western Australia, November 29%December 1, 1995.